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SUMMARY 

A  stochastic  model  for  paired  comparisons  of  multi -attribute  social  stimuli 
is  proposed  where  one  objective  is  to  find  the  relative  importance  of  the  attrib 
utes  for  a  judge  .  It  is  related  to  the  Bradley-Terry  model  where  log-parameters 
are  linear  combinations  of  functions  of  the  stimuli's  attributes  .  This  model 
does  neither  assume  that  the  functions  are  fixed  given  in  advance  nor  that  dif¬ 
ferent  judges  have  the  same  set  of  functions  .  The  choice  among  such  functions 
however  is  admitted  only  within  a  finite  given  scope  .  Within  the  framework  of 
exponential  families  maximum  likelihood  estimators  and  tests  are  derived  and  ap¬ 
plied  to  data  coming  from  a  psychological  experiment  with  schematic  faces  . 


Key  words  :  Paired  comparisons  ;  Social  Stimuli  ;  Exponential  families  ; 


Maximum  likelihood  estimators  . 
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1.  Introduction  . 


Almost  all  approaches  of  multi -attribute  decision  and  judgment  research 
consider  the  judment  U  as  a  function  of  the  attributes  JL k  that  character¬ 
ize  the  objects  G  : 

U  (  G(i)  )  =  F  [aj  fjfv#}1)) . aK  fK(^i))  ]  ,  (  1  ) 

where  fj  ,  ...  ,  f^  are  the  functions  which  represent  cognitively  the  K 

attributes  of  objects  s(') . G<N>  ,  and  where  a^  ,  ...  ,  a^  are  the 

weights  which  indicate  the  importance  of  the  attributes  ,  ...  ,  Jt ^  .  The 
attributes  jointly  describe  all  objects  by  different  aspects  .  F  is  a  real 
valued  monotone  function  in  each  of  its  arguments  and  specifies  the  composite 
effect  of  the  attributes  . 

If  the»^k's  have  scale  values  already  ,  e.g.  the  test  scores  of  intelli¬ 
gence  ,  experience  ,  or  neurotic ism  of  an  applicant  ,  if  the  f^'s  are  the  iden¬ 
tity  functions  ,  and  if  the  judgment  U  gets  a  value  on  the  judgment  criterion, 
e.g.  the  degree  of  the  applicant's  qualification  ,  weights  a^'s  can  be  simply 
estimated  from  models  ,  e.g.  from  the  model  of  multiple  linear  regression  (  cf. 
Darlington  (1968)  )  . 

In  many  situations  the  decision  between  two  objects  G^  and  G^  at  a 
time  which  one  of  them  is  more  attractive  ,  more  qualified  ,  or  rather  more 
threatened  ,  is  psychologically  more  meaningful  and  less  difficult  for  the 
judge  than  to  assign  a  number  on  a  rating  scale  .  Bradley  (1976)  has  given  an 
excellent  overview  over  the  methods  in  the  area  of  paired  comparisons  . 

If  the  attributes  ,  ...  ,  vHfK  have  scale  values  while  the  judgments 
result  from  paired  comparisons  ,  Srinivasan  and  Shocker  (1973)  have  proposed 
an  estimation  procedure  for  the  weights  aj  ,  ...  ,  a^  from  a  system  of  lin¬ 
ear  inequalities  . 
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However  ,  if  judgments  are  made  with  respect  to  paired  comparisons  of  social 
objects  ,  as  to  the  attractiveness  between  two  particular  persons  or  holiday 

places  ,  respectively  ,  models  for  the  estimation  of  importances  a^  .  aK 

should  satisfy  the  following  two  conditions  . 

[1]  It  cannot  be  assumed  that  the  attributes  of  the  objects  have  already  fixed 
functions  f j  ,  ...  ,  f^  being  available  for  the  judge  . 

Under  this  condition  ,  the  approach  of  Srinivasan  and  Shocker  (1973)  is  not 
appropriate  .  Basically  ,  the  attributes  could  have  got  such  functions  from  the 
judges  ,  as  it  is  assumed  in  several  approaches  .  However  ,  this  procedure  re¬ 
quires  two  conflicting  judgment  situations  within  the  same  judge  : 

(a)  The  decision  between  two  objects  (  stimuli  )  and  at  a  time  , 

and 

(b)  The  decision  for  scale  values  for  each  aspect  of  each  respective  attribute 
in  the  judgment  criterion  ,  e.g.  in  the  attractiveness  of  a  holiday  place. 

It  is  intuitively  clear  that  these  two  judgment  situations  will  mutually  in¬ 
fluence  each  other  and  thus  will  inflate  the  validity  of  the  model  . 

[2]  It  cannot  be  assumed  that  different  judges  decide  on  the  basis  of  the  same 

set  of  functions  f j  ,  ...  ,  f^  for  the  attributes^  ,  ...  of  the  objects. 

The  aspects  within  the  attributes  often  are  preferred  quite  differently  by  dif¬ 
ferent  judges  .  For  instance  ,  one  sea-side  resort  will  be  preferred  by  one  judge 
due  to  its  much  frequented  beaches  while  the  other  resort  will  be  preferred  by 
another  judge  due  to  its  tranquil  beaches  .  Averaging  over  judges  would  lead  to 
results  which  represent  no  one  of  the  judges  .  Thus  an  analysis  of  the  judgments 
of  the  individual  judge  should  be  performed  . 


A 


The  objective  of  the  present  paper  is  to  propose  a  stochastic  model  for  the 
decision  process  as  well  as  for  the  estimation  of  the  weights  ak  of  the  at¬ 
tributes  JC k  ,  k  =  1,  ...  ,  K  .  Its  applicability  will  be  demonstrated  with 
paired  comparisons  of  schematic  faces  in  a  psychological  experiment  .  The  final 
form  of  the  model  satisfies  both  conditions  [13  and  [2]  . 

In  the  second  section  normal  equations  for  the  maximum  likelihood  estimators 
will  be  derived  as  well  as  tests  for  hypotheses  with  respect  to  the  a^'s  .  In 
the  third  section  the  results  of  the  experiment  will  be  analysed  by  the  proposed 
procedures  .  The  results  corroborate  the  importance  of  condition  (23  . 


2.  The  Stochastic  Model  . 

Let  G^  ,  ...  ,  G^  be  N  stimuli  (  objects  )  which  can  be  described  by 

K  attributes  6  {l»  2»  •••  »  %}  with  fixed  mk  ,  k  =  1,  ...  ,  K  .  Thui, 

they  may  be  represented  by 

G(n)  =  ,  ...  ,^£n)  )  ,  n  -  1.  ...  ,  N  ,  N  =  If  mk  .  (2) 

Each  of  the  N(N-l)/2  pairs  of  stimuli  are  presented  in  a  random  succesion  to 
a  person  (  judge  )  who  has  to  decide  in  each  case  which  one  of  the  two  stimuli 
is  preferred  to  the  other  one  . 

To  fix  ideas  ,  let  us  assume  at  the  beginning  that  the  judge's  decision  proces 
can  be  described  by  the  following  model  ,  which  can  be  viewed  as  being  a  special¬ 
ization  of  the  classical  Bradley-Terry  (1952)  model  .  Here  the  N(N-l)/2  judgment 
are  carried  out  mutually  independently  with  probabilities  p^j(  £  )  :«=  pr  (  G^ 

is  preferred  to  G^)  )  which  depend  on  K  unknown  parameters  £  :=  (  aj  ,  ... 


a^  )  in  the  following  way  : 


5  - 


pu(a)  =  exp  { jC  >k  [hJ’1  -  ^j)]}/  (  1  *  exp  f  i  ak  [m<’>  -  N<J>]})  (  3  ) 

1  -  i  <  j  -  N  ,  where  ^  =  fk  ^  »  i  =  1,  . ..  ,N  ,  k  =  1,  ... ,K  , 

and  fj,  ...  ,  f^  are  fixed  known  real -valued  functions  . 

Let  the  outcomes  be  random  variables  X. .  g  {0  ,  1?  ,  where  X •  •  =  1  (  0  ) 

'  J  '  J 

has  the  meaning  that  G^'  is  preferred  to  (  G^  is  preferred  to 

Thus  the  likelihood  function  is  given  by 


pr  (  X.j j  =  x.j  ,  1  *  1  <  j  <  N  ) 

exp{^fak  ^  x,j  ["k*1  -  } 

.TT(leexp(g.k  [m<’).N(J)]}) 
xij  €  ^  0  »  1 3  »  1  "  1  <  J  ~  N  • 

It  can  be  seen  immediately  that  the  following  random  vector  I  is  a  suffi- 

K 

cient  statistic  for  a  €P  : 


Z  =  (  Z,  ,  . . .  ,  Zj,  )  » 


where  ZR  *  2Z1  X.j  [m^  -  M<j)  ]  ,  k  *  1,  ...  ,  K  . 


(  5  ) 


i  <  J 


Moreover  ,  the  distributions  of  Z  constitute  a  K  -  parametric  exponential 

1/ 

family  ,  since  for  every  fixed  a  €F 


pr  (  Z  =  z  )  =  exp  {  ak  zk  +  c(  a  )  +  d(  z  )  }  , 

I  s  (zl »  •  •  •  »z|^)  €  ^  » 

where  Z,  c{  a  )  and  d(  z  )  depend  on  f«  .....  in  an  obvious 


(  6  ) 


manner  . 


t 
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The  parametrization  is  identifiable  if  and  only  if  for  every  k  €  {1,  ...  ,K  j 
n>k  -  2  and  the  values  f |^(  1 )  ,  ...  ,  f^m^)  are  not  all  identical  .  Under 
this  condition  an  unique  maximum  likelihood  estimator  £  :=  (  ^  ,  ...  ,  aK  ) 
for  a  exists  if  the  following  system  of  equations  has  a  solution  : 

Lxi j  "  pij(  -  )  ]  [Mk  ^  "  Mk^  3  =  0  »  k  =  •••  »K  »  ( 

which  then  is  the  unique  solution  of  (  7  )  and  equals  to  £  . 

System  (  7  )  constitutes  a  system  of  normal  equations  which  result  from  put¬ 
ting  all  partial  derivations  of  the  likelihood  function  (  4  )  equal  to  zero 
and  some  additional  standard  analysis  .  That  (  7  )  in  fact  leads  to  maximum 
likelihood  estimators  can  be  seen  more  quickly  by  realizing  that  a  solution 

A 

a(  z  )  of  (  7  )  satisfies  the  canonical  conditions  for  exponential  families  : 

E  $(  z  )  (  Zk  )  =  zk  »  k  =  *  K  ’  (  8 

Obviously  ,  the  form  of  a  solution  of  (  7  )  in  case  of  her  existence  depends 
heavily  on  the  form  of  the  given  fj  .  ...  ,  fK  .  It  cannot  be  given  explicitly 
in  general  but  has  to  be  evaluated  numerically  in  every  concrete  situation  . 

Remark  1  :  If  not  all  N(N-l)/2  pairs  of  stimuli  but  only  a  fixed  subset  of 
them  are  presented  to  the  person  and  judged  by  her  ,  similar  results  can  be  de¬ 
rived  .  The  only  thing  that  changes  in  the  formulas  is  that  all  sums  and  pro¬ 
ducts  in  (  4  )  ,  (  5  )  and  (  7  )  have  to  be  restricted  to  this  subset  .  However, 
the  identifiability  of  the  parametrization  depends  now  additionally  on  this  sub¬ 
set  . 

Remark  2  :  If  the  N(N-l)/2  judgments  are  collected  independently  from  R 
persons  (  judges  )  who  can  be  assumed  to  judge  according  to  (  3  )  with  a  common 
^  ,  our  results  derived  so  far  hold  analogously  .  One  only  has  to  replace  the 
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X-^'s  and  x./s  throughout  by  the  arithmetic  means  of  the  corresponding 
responses  of  the  R  persons  and  to  pay  attention  to  the  fact  that  then  (  4  ) 
is  no  longer  the  likelihood  function  but  now  her  R-th  root  .  Especially  ,  the 
model  can  be  examined  with  generalized  (maximum)  likelihood  ratio  tests  in  two 
directions  which  can  be  established  by  use  of  the  well  known  *X.  -  approxima¬ 
tion  .  The  first  tests  the  hypothesis  :  "  a^  =  . . .  =  aK  =  0  "  (  pure  ran¬ 
domness  )  versus  K  :  "  model  (  3  )  "  ,  and  significance  supports  our  model  . 

The  second  tests  the  hypothesis  fT :  "  model  (  3  )  "  versus  ]<' :  "  Bradley- 
Terry  model  (1952)  "  ,  and  non-significance  then  supports  our  model .Of  course, 

one  should  clearly  distinguish  between  the  meaning  of  "  supports  "  in  the  two 
tests  due  to  the  well  known  unsymmetry  in  the  theory  of  testing  hypotheses  .  It 
should  be  pointed  out  also  that  in  case  of  R  =  1  person  both  tests  are  not  at 
hand  and  even  the  parameters  in  the  Bradley-Terry  model  (1952)  are  then  most¬ 
ly  not  estimable  within  the  maximum  likelihood  approach  . 

Instead  of  proceeding  along  the  lines  as  indicated  in  Remark  2  ,  we  prefer 

to  generalize  our  model  ,  admitting  a  more  individual  behavior  of  the  single  per¬ 
sons  .  By  this  we  do  not  only  mean  that  parameters  &  may  now  differ  from  person 
to  person  but  also  that, instead  of  having  _f  :=(  fj  .....  fK  )  fixed  known  , 
we  assume  now  that  only  f^  e  holds  ,  where  is  a  given  finite  set  of 

A  A 

real-valued  functions  ,  k  =  1,  ...  ,  K  .  Maximum  likelihood  estimates  f  :*  (  f,  , 

a  A 

...  ,  f^  )  and  &  for  a  single  person  are  determined  now  in  the  following  way  : 

For  every  fixed  f  with  f.  €  ,  k  *  1,  ...  ,  K  a  solution  of  (  7  )  has  to 

be  found  and  ,  together  with  f  ,  to  be  inserted  into  the  likelihood  function 

(  4  )  .  The  largest  value  of  (  4  )  then  determines  the  maximum  likelihood  esti- 
2  .  A 

mates  f  and  £  . 

As  long  as  the  sizes  of  the  sets  ^  ,  ...  ,  T'v  are  not  too  large  ,  the  fol¬ 
lowing  single  tests  for  every  fixed  f  can  be  combined  to  a  simultaneous  test  for 
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H  :  "  =  . . .  *  aK  =  0  "  .Its  p  -  value  can  be  bounded  from  above  in  the 

usual  manner  with  the  help  of  the  single  tests'  p  -  values  and  Bonferroni's 
inequality  .  Thus  .let  f  be  fixed  for  a  moment  and  let  us  look  for  a  test 
for  H  versus  the  general  alternative  that  JH  is  not  true  .  For  every  ke^l, 
,  K  ]  for  the  testing  problem  Hk  :  "  ak  =  0  "  versus  Kk  :  "  ak  t  0  " 
there  exists  an  UMPU  -  test  who  rejects  Hk  if  Zk  falls  outside  of  an  in¬ 
terval  .  But  the  boundaries  of  the  interval  depend  not  only  on  the  level  but 
also  on  the  values  of  Zj  ,  ...  ,  Zk_j  ,  Zk+j  .....  ZK  (  cf.  Lehmann  (1959) 
p.  134  )  .  Thus  in  view  of  the  combinatorial  difficulties  arising  from  f  , 
these  tests  are  practically  not  performable  in  general  . 


Alternatively  ,  let  us  propose  the  following  asymptotic  tests  for  H  (in¬ 
stead  of  Hk  )  versus  Kk  ,  k  =  1,  ...  ,  K  .  The  single  tests  hereby  are 
based  on  statistics 


5  kj  •  1/2J  K1’  -  Mkj|J  /  (»'♦  •  Mkj)  ] 2 ) 

•  *  J  ■  ^  J 


1/2 


a  ^ 


and  acceptance  regions  _1(*/2K  )  ,  J_1(  1  -  «</2K  )  J  ,  k  =  1 . K  , 

where  denotes  the  standard  normal  cumulative  distribution  function  and  °<  the 
level  of  the  corresponding  test  .  A  necessary  and  sufficient  condition  for  the 
asymptotic  normality  of  the  test  statistics  in  a  sequence  of  models  (  2  )  and 
(  3  )  (  triangular  array  )  is  that  for  large  N  the  following  terms  tend  to  0  • 


max 

r<s 


[Mkr)  -  Mks> 


]  2  /  XI  K’1  -  M«>]  2  ,  k  •  1 . K  .  ( 


n  > 


This  since  under  JH  the  ,  1  ^  i  <  j  ^  N  ,  are  independently  identically 
distributed  Bernoulli  -  variables  with  parameter  1/2  ,  and  therefore  the  condi¬ 

tions  are  equivalent  to  the  Lindeberg  condition  (  cf.  Feller  (1971)  p.264  (f)  )  . 


Remark  3  :  To  reduce  the  loss  of  power  induced  by  the  use  of  Bonferroni's  in- 
**qual ’  ,  other  testing  procedures  are  thinkable  ,  e.g.  those  which  are  based  on  the 
joint  normality  of  the  K  statistics  given  in  (  9  )  . 
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3.  Applications  to  a  Psychological  Experiment  . 

The  objects  (  stimuli  )  of  the  psychological  experiment  are  schematic  faces 
with  K  =  3  attributes  :  "  Mouth  "  €  £l  (  /\  )  ,  2  (  —  )  ,  3  (  v  )  , 

"  Havr  "  vX2e{  1  (  thin  )  ,  2  (  short  )  ,  3  (  full  )},  and  "  Eyes  " 

&  {l  (  —  )  »  2  (  •  )  }  ,  (see  Figure  1  )  . 


Please  insert  Figure  1  here 


Each  pair  of  the  N  =  18  faces  has  been  presented  to  the  person  (  judge  )  by 
projecting  the  two  corresponding  slides  onto  a  sreen  in  a  random  ordering  .  The 
person  decided  spontaneously  which  one  of  the  two  faces  looked  more  likable  by 
crossing  against  one  of  two  little  boxes  .  In  this  manner  N(N-l)/2  =  153  deci¬ 
sions  of  the  person  have  been  recorded  .  36  persons  have  participated  in  groups 

of  4  in  this  experiment  . 

To  get  the  numerical  analysis  manageable  we  restrict  ourselves  to  functions 
which  lead  to  equidistant  scales  .  Under  this  restriction  the  choice  of  the  ad¬ 
missible  functions  f^  ,  f2  ,  f^  can  be  restricted  to  the  following  classes  with¬ 
out  further  loss  of  generality  : 

=  {  p  ,  q  ,  r  }  where  p(  1  )  -  1  ,  p(  2  )  -  2  ,  p(  3  )  -  3 

q(  1  )  -  2  ,  q(  2  )  -  1  ,  q(  3  )  -  3 

and  r(l)  =  l,r(2)  =  3,r(3)  =  2, 

$2  a  3*1  and  =  {s}  where  s(l)=l,s(2)=2. 

For  each  fixed  k  e  {  1  ,  2  ,  3  }  the  justification  is  as  follows  :  In  the 
model  (  2  )  ,  (  3  )  the  -  values  appear  only  in  terms  of  differences  . 


Thus  their  smallest  value  can  be  set  equal  to  1  .A  t-  fold  of  these  dif¬ 
ferences  can  be  compensated  in  the  parameter  a^  by  a  change  to  a^/if  .  Thus 
their  largest  value  can  be  set  equal  to  3  .  Finally  ,  a  complete  reverse  of 

the  three  values  (  e.g.  (1,3,2)  to  (3,1,2)  )  can  be  compensated 
by  a  change  of  the  sign  of  a^  (  for  more  details  see  also  2)  below  )  .  The 
results  are  given  in  Table  1  . 


Please  insert  Table  1  here 


1)  The  estimated  functions  fj  and  f^  obviously  confirm  our  condition  [2]  . 

-  A  A  A 

A  comparison  of  the  ^ 's  among  persons  having  the  same  functions  f^  and 

and  signs  in  the  a^'s  (  cf.  2)  )  demonstrates  additional  individual  behavior  . 

2)  A  negative  sign  of  an  a^  means  that  the  person  shows  the  opposite  prefer¬ 

ence  fk  (  fk  =  h  «fk  with  h(  1)  =  3  ,  h(  2  )  =  2  ,  h(  3  )  =  1  )  to  f^ 
within  the  attribute  ,  k  =  1  ,  2  ,  3  .  The  absolute  values  of  the  para¬ 

meters  a^  ,  a2  ,  a^  of  a  person  indicate  which  relative  importance  the  three 

a 

attributes  exert  on  the  decision  process  .  A  negative  sign  of  an  ak  and  the 
absolute  values  of  the  a^  ,  ,  a^  are  thus  to  be  interpreted  analogously  . 

Remark  4  :  For  person  No.  7  maximum  likelihood  estimates  are  not  available  . 
The  data  reflect  a  deterministic  behavior  with  respect  to  JC.  and  neglect  of 
^2  and  ^  as  l°n9  as  tlie  tw0  faces  in  a  pair  differ  with  respect  to  . 

For  person  No.  17  both  q  and  r  are  maximum  likelihood  estimates  for  f2  , 

A 

and  the  sign  of  a2  depends  on  the  choice  among  them  .  For  person  No.  19  both 
p  and  q  are  maximum  likelihood  estimates  for  . 


3)  Column  "  Subj.  "  gives  the  subjective  rank  order  of  the  importance  of  the 

attributes  for  each  person  .  They  have  been  requested  in  interviews  following  the 
experiment  .  For  instance  ,  (1,3,2)  is  to  be  understood  that  the  person 

rated  as  most  and  as  least  important  for  her  judgements  .  If  a  per¬ 
son  rated  all  attributes  as  equally  important  this  is  denoted  by  a  (  -  ,  -  ,  - 

Remark  5  .  As  a  measure  of  the  difference  between  the  subjective  rank  order  of 

a  person  and  her  estimated  rank  order  (  cf.  2)  )  let  us  take  the  number  of 
inversions  .  Among  those  persons  who  could  give  a  subjective  rank  order  a  X  - 

test  with  3  degrees  of  freedom  can  be  performed  for  the  hypothesis  of  "  Random¬ 
ness  "  versus  the  alternative  that  less  inversions  are  more  likely  .  The  p  -  val¬ 
ue  for  these  30  persons  (  person  No.  7  is  treated  conservatively  )  is  p  = 

0.0268  . 

4)  The  last  column  gives  the  upper  bounds  for  the  p  -  values  of  the  proposed 
test  (  "  Randomness  "  versus  "  Model  "  )  for  every  person  .  Each  value  by 
Bonferroni's  inequality  is  calculated  as  the  7  -  fold  of  the  minimum  of  the 

p  -  values  of  the  single  tests  which  are  taken  from  Owen  (1962)  . 

Remark  6  .  The  numerical  calculations  have  been  performed  on  the  H  B  66/80 

of  the  University  of  Mainz  .  The  solutions  of  the  system  of  equations  (  7  )  are 

based  on  a  procedure  of  Werner  (1979)  . 
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Table  1 


Estimates,  subjective  rank  orders,  and  p-values  of  36  persons 


Person 

«*, 

•i2) 

*1 

S2 

*3 

Sub  j . 

Test 

1 

(P 

#P) 

8.7230 

2.7481 

1 .2790 

(3,1,2) 

5.60- 

-2 1 
10  z 
-1  A 

2 

(P 

*P> 

5.5402 

1 .8673 

1  .7035 

(2,1,3) 

1.59- 

o  o  o 

1  1 

3 

(P 

*q> 

4.2700 

1 .5842 

1.5627 

(1 ,3,2) 

5.59- 

4 

(P 

>q) 

2.8581 

1.1590 

1.6735 

(3,1,2) 

9.54- 

5 

(P 

fP> 

3 . 2006 

1.1475 

1  .0484 

(3,1,2) 

3.13- 

10_U 

10*1'' 

6 

(q 

fP) 

1 .8752 

1 .0757 

0.6223 

(3,2,1) 

4.22- 

7 

(p 

.-) 

CO 

— 

— 

(3,1 ,2) 

5.60- 

io-21 

io"1' 

8 

(q 

/p) 

2.2223 

0.7462 

1.5745 

(2,1,3) 

7.33- 

9 

(p 

#q) 

4.6310 

1 .8901 

0.4377 

(3,2,1) 

2.51 

io”1s 

10 

(p 

>9) 

2.4133 

0.7431 

2.0370 

(1,3,2) 

2.88- 

io'1c 

1 1 

(p 

/P) 

6.3412 

2.4086 

0.4456 

(-,-,-) 

3.82- 

io”20 

12 

(p 

>p) 

2.4624 

2.5545 

0.7648 

(2,1,3) 

2.88- 

io”1c 

13 

(p 

;r) 

1 .6597 

-1 .9288 

1  .0031 

(-,-,-) 

3.96 

io'9 
-1  6 

14 

(p 

/q) 

3.8592 

0.9703 

1.6856 

(2,1,3) 

3.13- 

10 

15 

(p 

rq) 

2.9308 

-0.2548 

1.4654 

(3,1,2) 

8.71- 

io“15 

16 

(q 

/r) 

0.7244 

-0.3808 

1 .0754 

(2,1,3) 

1  .09- 

io'9 

17 

(p 

,q  or 

r) 

1 .7691 

±1.6961 

0.8527 

(-,-,-> 

2.55- 

io"8 

18 

(q 

,r) 

1 .4558 

-1  .0516 

1 .3633 

(1,3,2) 

1 .39* 

io-8 

19 

(p 

fP  or 

q) 

1.1351 

0.9076 

0.1316 

(2,1,3) 

8.39 

10~8 

20 

(p 

,r) 

1 .7985 

-0.6672 

1.2310 

(-,-,-) 

1  .09 

10~9 

21 

(p 

rP) 

1 .8366 

1 .8763 

1 .0929 

(3,2,1) 

4.65 

io-8 

22 

(p 

fP) 

2.6216 

2 . 3006 

0.8293 

(1 ,3,2) 

7.33- 

io-11 

23 

(p 

rP) 

2.9433 

1 .9670 

Q.6170 

(2,3,1) 

4.34- 

io-14 

.  -1 3 

24 

(p 

fP) 

1 .6562 

-0.4739 

0.2743 

(3,1,2) 

2.Q8- 

10 

25 

(p 

fP) 

2.3590 

0.6951 

0.5175 

(3,1,2) 

3.13- 

io-16 
-1 1 

26 

(p 

,q) 

2.1148 

1.8883 

0.3272 

(2,3,1) 

7.33- 

10 

27 

(q 

#q) 

0.8842 

1.1139 

0.3190 

(1,3,2) 

2.66- 

io”7 

28 

(p 

^p) 

1.2671 

0.2191 

0.3805 

(3,1,2) 

2.88' 

10‘10 

29 

(p 

2.6836 

-0.1918 

1  .3614 

(2,1,3) 

8.71- 

io”1 " 

30 

(p 

*p) 

2.5130 

0.3020 

0.7664 

(3,1,2) 

3.13- 

io”16 

31 

(p 

,q) 

0.6130 

1.8514 

2.5592 

(1,2,3) 

1 .69- 

io”15 

32 

(p 

fr) 

1.3106 

-1.2606 

2.4690 

(1,2,3) 

1  .69 

io”15 

33 

(p 

>q) 

0.1412 

0.4580 

-0.2010 

(-,-,-) 

1  .36- 

io'“ 

34 

(r 

#r) 

-0.8450 

-1.4463 

-0.1024 

(-,-,-) 

7.33 

io”11 

35 

(p 

rr) 

3.0601 

0.4474 

2.6794 

(2,1,3) 

4.22 

io"'2 

36 

(p 

/P) 

1 .6102 

0.3132 

0.4356 

(3,1,2) 

9.54 

io”  3 
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